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Research in information security has generally focused on providing a comprehensive interpretation 
of threats, vulnerabilities, and attacks, in particular to evaluate their danger and prioritize responses 
accordingly. Most of the current approaches propose advanced techniques to detect intrusions and 
complex attacks but few of these approaches propose well defined methodologies to react against a 
given attack. In this paper, we propose a novel and systematic method to select security counter- 
measures from a pool of candidates, by ranking them based on the technical and financial impact 
associated to each alternative. The method includes industrial evaluation and simulations of the 
impact associated to a given security measure which allows to compute the return on response in- 
vestment for different candidates. A simple case study is proposed at the end of the paper to show 
the applicability of the model. 

Keywords. Cyber Protection Level, Countermeasure Selection, Complex Attacks, Security Met- 
rics, Decision Support. 



1 Introduction 

Innovation in Information Technology has brought numerous advancements but also some consequences. 
Cyber attacks have evolved along with technology, reaching a state of high efficiency and performance. 
Current research focuses on approaches to detect such attacks and demonstrate their strengths and the 
difficulty to mitigate them HJI21- Most of these works propose approaches to detect complex attacks but 
few of them propose a methodology to react against them. 

In addition, research in dynamic response proposes automatic response mechanisms (e.g., the adap- 
tation of security policies) to overcome the limitations of manual responses. However, these approaches 
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remain limited since they do not analyse the impact of the selected countermeasures (3)- Inappropriate 
selection of countermeasures result in disastrous consequences for the organisation. An impact analysis 
of all the security candidates is therefore essential in the decision process to select appropriate counter- 
measures for a given attack. 

In this paper, we propose a novel and systematic method to select security countermeasures from 
a pool of candidates, by ranking them based on the trade-off between their efficiency in stopping the 
attack, and their ability to preserve, at the same time, the best service to legitimate users. The method 
includes industrial evaluation and simulations of the impact associated to a given security measure. 

The rest of the paper is structured as follows: Section |2] introduces the state of the art on service 
protection level, and return on response investment. Section [3] presents the security and financial coun- 
termeasure impact analysis. Section 0] details the quantification of the proposed countermeasure impact 
model. Section[5]gives an example of a case study. Finally, conclusions and perspectives for future work 
are presented in Section [6] 



2 State of the Art 

2.1 Cyber Protection Level 

The cyber protection level is an evolution of the safety integrity level (SIL) (UHl, which is defined as a 
relative level of risk reduction provided by a safety function. A SIL is determined based on a number of 
quantitative factors (using methods such as: risk matrices, risk graphs, layers of protection analysis) in 
combination with qualitative factors such as development process and safety life cycle management. 

The cyber protection level refers to the strength of cyber security means deployed against a particular 
threat. The process is generally used to identify assets, threats, vulnerabilities, likelihood, countermea- 
sures, and consequences. This is usually obtained from a risk analysis, following any of the international 
standards (e.g., NIST (6), ISO 0), or any of the risk management methodologies (e.g., MEHARI (8) 
and EBIOS (9l) as well as expert knowledge. In our study, we consider the EBIOS methodology, defined 
by the French National Security Agency (ANSSI)® . 

The analysis follows several steps: 1) the context definition that determines stakeholders, processes, 
assets and dependencies, threat sources and existing security; 2) feared events and threat scenarios, 
with impact and occurrence probability; 3) risk evaluation that takes into account the existing security 
described in (1); 4) necessary measures related to risk mitigation. 

Although most organizations follow a particular methodology to deploy a risk analysis, current ap- 
proaches present several shortcomings: they rarely propose calculation methods for the protection level; 
none of them can be applied on an operational environment with “living” protection means (i.e., poten- 
tially unavailable for a period of time); they do not consider the different instances that must be deployed 
in the network to cover the threat everywhere; the effectiveness of a protection function is hardly consid- 
ered in the analysis. 



2.2 Cost Sensitive Metrics 

Cost sensitive metrics are widely proposed as a viable approach to find an optimal balance between 
intrusion damages and response costs, and to guarantee the choice of the most appropriate response 
without sacrificing the system functionalities. 
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2.2.1 Return On Investment (ROI): 

The simplest and most used approach for evaluating financial consequences of business investments, 
decisions and/or actions is the ROI metric. ROI is a relative measure that compares the benefits versus 
the costs obtained for a given investment ifTOlfTTll . 

ROI basically shows how much a company earns from invested money. This metric supports decision 
makers in selecting the option(s) that have the highest return. ROI is calculated as the present value of 
accumulated net benefits over a certain time period minus the initial costs of investment, then divided by 
the initial costs of investment, as shown in Equation [B 



Where: 

B t refers to all benefits during period t, 

C f refers to all costs during period t 

The decision rule is that the higher the ROI value, the more interesting the investment. However, 
Jeffery flOll agrees that the major problem with ROI is that the metric does not include the time value of 
money, i.e., a 100% ROI realized 1 year from today is more valuable than a 100% ROI realized over 5 
years. Furthermore, the costs and benefits of the project may vary over time, meaning that the cash flows 
are different in each time period. As a result, ROI is not a convenient way to compare projects when 
the costs and benefits vary with time, and it is also not useful for comparing projects that will run over 
different periods of time. 

2.2.2 Return On Security Investment (ROSI): 

It is a relative metric that compares the differences between the damages caused by attacks (before and 
after countermeasures) against the cost of the countermeasure H2l[T4l|T3l. To calculate ROSI, a formula 
adapted from the ROI metric is presented in Equation [2j 



Where: 

ALE/, refers to the annual loss expectancy before countermeasure, 

ALE a refers to the annual loss expectancy after countermeasure, 

Cost cm is the cost of the countermeasure 

The calculation of each parameter composing the ROSI equation has been widely discussed by Lock- 
step Consulting fl2l . and Kosutic lfl3ll . The former proposes a methodology that considers different 
levels of li kelihood and severity, which are then, respectively transformed into frequency and direct cost; 
the latter considers on the one hand, parameters associated to the incident (e.g., financial losses, costs, 
frequency), and on the other hand, parameters associated to the protection (e.g., cost, benefits, life ex- 
pectancy of the security measure). 

Similar to the ROI metric, the decision rule states that the higher the ROSI value, the more interesting 
the investment. 



ROI = Bt c Ct x 100 



( 1 ) 




( 2 ) 
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2.2.3 Return On Response Investment (RORI): 

It is a service dependency index for cost sensitive response based on a financial comparison of the re- 
sponse alternatives lfl5ll . RORI is an adaptation of the ROSI index lfl4l that provides a qualitative compar- 
ison of response candidates against an intrusion. RORI considers not only response effects on intrusions, 
but also response collateral damages as depicted in Equation [3] 



RORI = 



[rc b - rc] - oc 

CD + OC 



x 100 



( 3 ) 



Where: 

lC b represents intrusion impacts when no response is enforced, 

RC refers to the combined impact of intrusion and response, 

OC are operational costs that cover low level investments such as response setup and deployment costs 
CD refers to collateral damages, which are costs that are added by a new response, and are not related to 
intrusion costs 



The deployment of the RORI index into real world scenarios has presented the following shortcom- 
ings: 

• The RORI index is not defined when no countermeasure is selected. Since the operational cost 
(OC) is associated to the security measure, the RORI index will lead to an indeterminacy when no 
solution is enforced (NOOP). 

• The RORI index is not normalized with respect to the size and complexity of the infrastructure. 

• The absolute value of parameters such as IC b and RC is difficult to estimate, whereas a ratio of 
these parameters is easier to determine, which in turn reduces errors of magnitude lfl6l . 

3 Countermeasure Impact Analysis 

3.1 Security Impact 

Taking into account current shortcomings in risk analysis methodologies, we propose to evaluate a pro- 
tection level against a threat, related to confidentiality, integrity, and availability, that considers technical 
ans business services as assets. 

The protection level (PL) of a service Sj against a threat 7) is calculated using Equation |U 

PL(Sj,T k ) = 100 - mox(0,AD - AP ) ( 4 ) 

Where: 

• AD = Assessed Danger, which represents the threat dangerousness in terms of confidentiality, 
integrity and availability (i.e., dck, djk, d A k), as well as the service value in terms of confidentiality, 
integrity and availability (i.e., vcu vn, vm), as shown in Equation [ 5 ] 

AD = ( d ck x vq) + (d,k x vn) + ( d Ak x v Ai ) * 10Q ^ 

From Equation |5j dck x v ci represents the confidentiality-related impact, djk x vn represents the 
integrity-related impact, and d A k x vm represents the availability-related impact. The more dan- 
gerous the threat and/or the more important the service in terms of confidentiality, integrity, and 
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availability, the higher the impact. Values of dangerousness d and service v range from 0 to 5, 
therefore, the maximum possible value for AD is 75. We multiply this result by 100/75 in order to 
get homogeneous values of protection (assessed in a scale from 0 to 100). 

The current proposal only considers the CIA services (i.e., confidentiality, integrity, and availabil- 
ity) for the calculation of the assessed danger, mainly due to two reasons: firstly, because the data 
is considered by the EBIOS methodology, and secondly, because the approach is used by our in- 
dustrial partners at Cassidian Cybersecurity. However, it is also possible to use other parameters 
(e.g., criticality, accessibility, recuperability, vulnerability) as long as we are able to estimate their 
values for the selected threats and services. 

• AP = Assessed Protection, which represents the protection assigned against the threat k for the 
service i (i.e., pik), as well as the effectiveness of the protection assigned against the threat k for 
the service i (i.e., e^), as shown in Equation [6] 

AP = e ik x p ik (6) 

From Equation |6l the assessed protection is estimated by experts. The effectiveness factor is 
calculated depending on the type and distribution of protection (e.g., false positive rates, coverage of the 
network, feedback from operational teams, subjective figure). 

To measure the impact of changes on security (SI, for Security Impact), we compare current and 
potential situations as depicted in Equations [7] and [8] 

SI = PL potential (Si ,T k )- PL currmt (Si , T k ) (7) 

Where, 

• PLpotential is the protection level with a modified protection capacity against the threat 7). 

Equation H] provides more details about the delta between the current and the potential situation. 

SI= max (O, [ fexva)+(^)+(^xv Ai ) x |QQ j _ x _ 
max (0, p d «xv a )+fexv 7 ,)+(^xv Ai ) x 1(X) ] _ ^ x p t^ 

( 8 ) 



The variation is being on the protection p% and/or its effectiveness e^. 



3.2 Financial Impact 

An improvement of the RORI index has been proposed, taking into account not only the countermeasure 
cost and its associated risk mitigation, but also the infrastructure value and the expected losses that may 
occur as a consequence of an intrusion or attack fl6l . 

The improved RORI index handles the choice of applying no countermeasure to compare with the 
results obtained by the implementation of security solutions (individuals and/or combined countermea- 
sures), and provides a response that is relative to the size of the infrastructure. The improved Return on 
Response Investment (RORI) index is calculated according to Equation [9] 



RORI = 



(ALExRM)-ARC 



100 



(9) 



Where: 



ARC+AIV 
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• ALE is the Annual Loss Expectancy and refers to the impact cost obtained in the absence of 
security measures. ALE is expressed in currency per year (e.g., $/year) and and includes loss 
of assets (La), loss of data (Ld), loss of reputation (Lr), legal procedures (Lp), loss of revenues 
from existing clients or customers (Lrec), loss of revenue from potential clients (Lrpc), other 
losses (01), contracted insurance (Ci), and the annual rate of occurrence (ARO), i.e., ALE = 
(. La + Ld + Lr + Lp + Lerc + Lrpc + Ol — Ci ) xARO 



► RM refers to the Risk Mitigation level associated to a particular solution. RM is defined from the 
security impact as a percentage (i.e., 0% < RM < 100%) that represents the additional counter- 
measure effectiveness over the best solution to be implemented in order to totally eradicate the 
threat. RM includes the protection level of potential and current situations, as presented in Section 
lollc „ mv/r_ PL potential(Si,T k )-PL current (Si,T k ) \ 

[3JJ(i.e., RM loo— PL curren , (Sj,T k ) ) 



• ARC is the Annual Response Cost that is incurred by implementing a new security action. ARC 
= OC+CD from Equation [3] ARC is always greater than or equal to zero ( ARC > 0), and it is 
expressed in currency per year (e.g., $/year). ARC includes Direct costs: e.g., Cost of imple- 
mentation (Ci), Cost of maintenance (Cm), Other direct costs (Ode); and Indirect costs (Ic), i.e., 
ARC = Ci + Cm + Ode + Ic 



• AIV is the Annual Infrastructure Value (e.g., cost of equipment, services for regular operations) 
that is expected for the system, regardless of the implemented countermeasures. AIV is greater 
than zero ( AIV > 0), and it is expressed in currency per year (e.g., $/year). AIV includes the 
following costs: Equipment Costs (Ec), Personnel costs (Pc), Service costs (Sc), Other costs (Oc), 
and Resell Value (Rv), i.e., AIV = Ec+Pc + Sc + Oc — Rv 

4 Countermeasure Impact Quantification 

The quantification of the parameters composing the RORI model proposed in Equation [9] is a task that 
requires expert knowledge, statistical data, simulation and risk assessment tools. Our experience in 
quantifying impact losses, as well as countermeasure costs and benefits for different security systems 
demonstrate that within 3 to 4 hours of discussions with use case providers and simple simulation runs, 
we are able to estimate each parameter composing the RORI model. The remaining of this section 
proposes a simple and well structured methodology to help security analysts in the estimation of such 
parameters. 

4.1 Annual Loss Expectancy 

For the estimation of the ALE, we adopted the approach proposed by Lockstep l[T2l to use the severity 
scale of values, which convert qualitative estimations into quantitative values of costs. For instance, a 
‘minor’ loss of assets (La) represents a cost of $1,000; whereas a ‘serious’ loss of assets (La) represents 
a cost of $1,000,000. The estimation of all other losses (i.e., Ld, Lr, Lp, Lrec, Lrpc, 01) follows the same 
approach. 

The likelihood of an incident is transformed into a frequency value, which results into the Annual 
Rate of Occurrence (ARO) parameter. For instance, a Tow likelihood’ means that the incident is likely 
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to occur once every year, (ARO = 1); whereas, a ‘high like li hood’ means that the incident is likely to 
occur once per month or less, (ARO = 12). 

Both parameters (i.e., severity and likelihood) are estimated using a survey and scoring system, which 
combine expert knowledge and statistical data to quantify risk exposure. In order to handle uncertainty, 
we use Monte Carlo simulation. To run our simulation, we chose triangular distributions to evaluate 
the most likely values assigned to each level of security and likelihood, with minimum and maximum 
possible values of each level. This type of statistical computations can be easily achieved using basic 
statistical software or spreadsheet cditorffl. 

After 250 iterations, we were able to obtain a value of the losses and frequency that compose the 
ALE parameter, which represents the expected annual loss as a consequence of the realization of a given 
threat. 



4.2 Risk Mitigation 

A risk analysis, as performed by Cassidian cyber-security expert^, gives the list of threats directly en- 
dangering business and technical services of the entity to protect, and the available protection means. 
The level of protection related to a given set of services is assessed using different kinds of i nf ormation: 

1 . Types of security devices able to detect and/or react against an activity related to a threat occur- 
rence; given by cyber-security experts. 

2. Instances of security devices actually deployed to protect services; given by security architects 

Services are modelled using dependency models. Identified threats and related protection measures 
(if they exist) are associated to services. We obtain, for each service a list of couples (threat, protection). 

A threat is characterized by a dangerousness level in terms of confidentiality (i.e., dck), integrity 
(i.e., dik) and availability (i.e., dU*)- We consider each service and their value as per confidentiality (i.e., 
va ), integrity (i.e., v/,) and availability (i.e., vm) in order to determine the potential effect of threats on 
services. An example of the asset values is represented in Tabled] 

Dangerousness levels and values are integers ranging from 0 (meaning respectively no danger / no 
value) to 5 (meaning respectively highest danger / biggest value). Dangerousness and asset values are 
given by experts. Cell values are calculated using the AD such as described in Equation [5] Highest threat 
effect would be 75 (dot product of danger level and service value per CIA criteria). The result is finally 
reported as a percentage. It is important to note that the “N/A” flag in some cells means the threat does 
not endanger the service. 

A protection (/?,*) is characterized by its effectiveness (e#) to prevent a threat from occurring. This 
is an integer ranging from 0 to 100. The protection either exists ( p % = 1) or does not exist (/?,* = 0). 
When the protection is different from 0, the related threats are supposed to be mitigated by some of the 
protection means described in the service model. 

Table [2] depicts an example of protection capacity on different services affected by several threats. 
We identify the services at which protection measures have been deployed, and their ability to mitigate 
threats. As a result, we consider the actual danger being the difference of threat level and protection 
level. Results of the aforementioned example are depicted in Table [3] 



'Quadrant: The Quick and dirty risk analysis tool, available at: www.qdmt.com/home.htm 

2 Monte Carlo simulation for excel featuring distribution strings, available at: http://xlsim.com/xlsim/index.html 

3 http://www.cassidiancybersecurity.com/en_US/web/guest/cybersecurity 
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Table 1 : Assessed Dangerousness Matrix 



c 

I 

A 

C I A 


5 


5 


0 


4 


3 


5 


5 


0 


4 


3 


5 


5 


2 


4 


3 


Service 1 


Service2 


Service3 


Service4 


Service5 


1 


2 


3 


Threat 1 


40 


40 


8 


32 


24 


3 


3 


3 


Threat2 


60 


60 


N/A 


N/A 


N/A 


2 


2 


2 


Threat3 


N/A 


N/A 


5 


32 


24 


5 


5 


5 


Threat4 


N/A 


100 


N/A 


N/A 


N/A 


4 


4 


4 


Threat5 


N/A 


N/A 


N/A 


N/A 


48 


5 


5 


5 


Threat6 


100 


100 


13 


80 


60 


3 


3 


3 


Threat7 


60 


60 


8 


36 


36 


2 


2 


0 


Threat8 


N/A 


27 


0 


N/A 


N/A 


4 


5 


3 


Threat9 


80 


N/A 


N/A 


N/A 


N/A 


3 


3 


3 


ThreatlO 


60 


60 


8 


48 


36 



Table 3: Actual Danger Matrix 
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A 

C I A 
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5 


2 


4 


3 


Service 1 


Service2 


Service3 


Service4 


Service5 


1 


2 


3 


Threatl 


40 


40 


8 


32 


24 


3 


3 


3 


Threat2 


-15 


-15 


N/A 


N/A 


N/A 


2 


2 


2 


Threat3 


N/A 


N/A 


-55 


-28 


-36 


5 


5 


5 


Threat4 


N/A 


100 


N/A 


N/A 


N/A 


4 


4 


4 


Threat5 


N/A 


N/A 


N/A 


N/A 


8 


5 


5 


5 


Threat6 


0 


0 


-87 


-20 


-40 


3 


3 


3 


Threat7 


10 


60 


-42 


36 


-14 


2 


2 


0 


Threat8 


N/A 


27 


0 


N/A 


N/A 


4 


5 


3 


Threat9 


80 


N/A 


N/A 


N/A 


N/A 


3 


3 


3 


ThreatlO 


-30 


-30 


-82 


-42 


-54 



Table 2: Protection Capacity 
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Service2 


Service3 


Service4 


Service5 


Threatl 












Threat2 


75 


75 








Threat3 






60 


60 


60 


Threat4 












Threat5 










40 


Threat6 


100 


100 


100 


100 


100 


Threat7 


50 




50 




50 


Threat8 












Threat9 












ThreatlO 


90 


90 


90 


90 


90 



Empty cells from Table [2] mean that protection does not exist in such service against a particular 
threat. Cells from Table [3] show the actual danger. In order to obtain the PL value in each cell, we use 
Equation |4] 
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4.3 Annual Response Cost 

In contrast to the AIV parameter, the ARC is a variable cost associated with the implementation of a 
given countermeasure. For instance, let us suppose that the user authentication information of a Web 
service is stored in a database. Whenever users want to access the system, they need to provide their 
corresponding login and password. However, for suspicious users, the organization wants to implement 
a countermeasure that asks for a double authentication (e.g., a challenge question, a security pin). The 
implementation of this countermeasure requires the organization to spend additional employee-hours 
which in turn represents a given cost. This latter is defined as the cost of implementation (Ci). 

In addition, the countermeasure is going to be active only for suspicious users for a given period 
of time, which means that the system will turn the countermeasure from ‘on’ to ‘off’ according to the 
security tests and analysis performed. These tests and analysis represent the cost of maintenance (Cm) 
to the organization. 

The activation/deactivation of a given countermeasure engenders other direct and indirect costs. For 
instance, requesting an additional authentication method to legitimate users may cause these users to 
unsubscribe from the service and search for another one. This collateral damage represents an indirect 
cost (Ic) to the organization. Collateral damages can be quantified as the variation between the current 
and the projected productivity that an organization experiences due to a side effect of a given solution 

ns. 



4.4 Annual Infrastructure Value 

This parameter is calculated as the sum of the annual value of all the equipments, i.e., Policy Enforcement 
Points (PEP), that are needed to be deployed in the preliminary phase of the system architecture in order 
to guarantee a desired level of security. The AIV includes the cost of purchasing, licensing, and/or 
leasing the security equipments in a given organization. 

It is important, however, to answer the following questions while estimating the AIV parameter: 

• What kind of PEPs (e.g., Firewalls, IPS, IDS, SIEM) and which quantity is required for the system 
security? 

• What is the lifetime expectancy of the PEP? 

• What is the PEP’s deployment time? 

• What is the annual cost of purchase, licensing or leasing of the PEP? 

• How many employee-hours are required for the operation of the PEP? 

• How long (i.e., hours/year) is the PEP expected to be active? 

• Is there an insurance contracted for the PEP? If so, how much does it cost per year? 

• How frequently (i.e., times/year) does the PEP need to be checked or maintained? 

• Is there any other cost associated with the operation of the PEP in the security infrastructure? 

• What is the amortization value of the PEP? 



5 Use Case 

This section describes a simple case study provided by Cassidian Cyber Security, the cyber security com- 
pany of the Airbus group, and a major provider of global security solutions and services. 
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The scenario is based on a risk analysis performed on a company, limited to three services. The 
security auditors have determined the value of these services for the company, taking into account the 
company activity, stakeholders, technical and human constraints (e.g., skill level of the personnel in terms 
of security-related good practices), the loss of money in case of failures, etc. The risk analysis has been 
performed according to the EBIOS methodology. Four threats have been considered in this study. Their 
effects on targeted elements enable the auditors to evaluate the dangerousness criteria. Countermeasures 
have been proposed by the auditors to make the risk level acceptable along the company criteria. 

The subsequent deployment of security devices compliant with the experts recommendations leads 
to provide the following matrices : threat target matrix (Table 0]), and protection capacity matrix (Table 



Table 4: Threat Target Matrix 



c 
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A 

C I A 
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5 


5 
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Web services 


Network infrastructure 


User service 


1 


3 


2 


Web site 

sabotage 


33 


N/A 


N/A 


3 


1 


5 


Network in- 
frastructure 
attack 


N/A 


60 


N/A 


5 


4 


2 


User work- 
station 
compromise 


N/A 


N/A 


68 


5 


4 


3 


Admin 

workstation 

compromise 


N/A 


N/A 


72 



Table 5: Protection Capacity Matrix 





Web services 


Network infrastructure 


User service 


Web site 

sabotage 


50 






Network in- 
frastructure 
attack 




80 




User work- 
station 
compromise 






17 


Admin 

workstation 

compromise 






50 



The main danger on user and admin workstations lies in their compromission by malware programs. 
To counter this threat we deploy a protection with an effectiveness assessed by experts e = 50%. The 
effectiveness value is obtained considering several criteria: 

• reliability of the malware detection software: the cyber company leading audits maintain a knowl- 
edge base regarding the reliability of security products. Particularly, anti-virus system reliability 
has been tested against malwares discovered and published within a period of 6 months. These 
tests are possible using online services such as YirusTotal. With an up-to-date base, 80% of the 
injected malware programs were detected by the malware detection tool deployed in the audited 
company. Then reliability score is 80%. 
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• signature base update policy: the frequency is set to one per week, which is assessed being far 
from achieving complete protection, therefore the score is set to 60%. The following scale is being 
used: 100% daily update, 60 % weekly update, 20% monthly update, 5% annual update, 1% no 
update since installation. 

• residence : this one is 100%, as tests over a 1 month period do not reveal any dysfunction. Indeed 
the cyber security company periodically launches test campaigns on security product resilience. 

The effectiveness is then evaluated as the product of the reliability, policy and resilience scores (i.e., e = 
reliability_score x update_policy_score x resilience_score). This gives a result of 48%, approximated 
to 50%. This kind of protection is deployed on every administration workstation, and in only 1/3 user 
workstations (900 PC among 2700 for the whole organization), mainly for cost reasons. The protection 
level (PL) is calculated using Equation @1 as follows: 

PL (User service, User workstation compromise) = 100 - (68 - 17) 

PL (User service, User workstation compromise) = 100 - 51 = 49 

A malware is detected on a user computer (among those unprotected). The proposed countermeasure 
consists on deploying an anti-malware agent on it and extend the solution to the other 1,800 workstations. 
The technical assessment of the countermeasure is shown in Tables [6] and [7] 
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Table 7: Potential Danger Matrix 
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The risk mitigation RM (user workstation compromise) = (82-49) / 51 = 65%. The anti-malware 
editor cost policy is the following: 40,000€ per year for a maximum of 2,000 embedded agents; 50,000€ 
per year for a maximum of 5,000 embedded agents. 
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Table 8: Countermeasure Evaluation against Malware Infection 



Countermeasure 


ARC 


RM 


RORI 


Cl. No operation (NOOP) 


0 € 


0.00 


0.00% 


C2. Install agent only in the infected hosts 


17 € 


0.01 


1.31% 


C3. Install agents in 1,100 hosts (to reach the 
2000 agent-limit) 


18,700 € 


0.39 


21.66% 


C4. Install agents in 1,800 hosts (to protect 
all workstations) 


40,600 € 


0.65 


21.11% 



Considering that the annualized loss expectancy for a malware in the system is estimated at ALE = 
100,000€ per year, and that the annual infrastructure value is estimated at AIV = 75,000€ per year, 
we use Equation [9] to perform the countermeasure evaluation. Table |8] shows these results, and details 
information regarding countermeasure cost, mitigation level, and RORI index. 

From Table[8l the first candidate (i.e., Cl) proposes to accept the risk by doing no operation (NOOP). 
This alternative does not provide any mitigation level (RM=0) and does not generate any additional cost 
(ARC=0). The expected return on the response investment is therefore null (RORI=0). 

The second alternative (i.e., C2) proposes to install agents only in the infected host. This alternative 
will not change the danger of the total group of 2,700 hosts. The mitigation level will be therefore close 
to zero (RM=0.01). However, taking into account that a license to install an anti-malware agent for a 
maximum of 2,000 hosts is already being paid, the ARC value to be installed in 1 additional host will 
only consider the cost of deployment (e.g., deploying a license in one host takes in average 10 minutes, 
and 1 employee-hour costs 100€ at Cassidian Cyber Security), therefore ARC(1 host) = 17€. 

The third alternative (i.e., C3) suggests to install agents in 1,100 additional workstations (the maxi- 
mum number of hosts allowed by the license). The mitigation level is calculated considering the current 
protection level ( PLcurrent = 49), and the potential protection level ( PLpotentiel = 100- max (0, 68- 
50*2000/2700) = 69), therefore RM = (69 -49) / (100 -49) = 39%. In addition, the ARC for 1,100 
additional hosts (to reach the 2,000 agents limit) is equivalent to ARC= 1100 x 17 = 18700 €. As a 
result, the return on response investment is equivalent to (RORI = 21.66%). 

The fourth evaluated candidate proposes to install agents in every administration workstation of 
the whole organization (i.e., 2,700 workstations). This requires to pay an additional of 10,000€, for a 
license that will allow to install agents in a maximum of 5,000 hosts. The mitigation level is calculated 
considering the current protection level ( PLcurrent = 49), and the potential protection level {PLpotentiel 
= 100- max (0, 68-50*2700/2700) = 82), therefore RM = (82 -49) / (100 -49) = 65%. In addition, the 
ARC for 1,800 additional hosts (to reach the 2,700 agents) is equivalent to ARC= 1800 x 17 = 30,600€+ 
10,000€= 40,600€. As a result, the return on response investment is equivalent to (RORI = 21.11%). 

After the evaluation of the different candidates to mitigate a malware infection, we select alternative 
3 as the optimal countermeasure, since it provides the highest RORI index. C3 proposes to install anti- 
malware agents in 1,100 hosts, additional to the already 900 protected hosts. 



6 Conclusions and Future Work 

We have proposed in this paper a novel and well structured method to select security countermeasures 
from a pool of candidates, based on their technical and financial impact. The method includes industrial 
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evaluation and simulations of the impact associated to a given security measure. 

By calculating the potential new protection level, we are able to compare the current versus the po- 
tential change. As a result, we obtain quantitative information on the improvement or degradation of 
security at the service level. However, nowadays this function is limited to the protection level mea- 
surement after the addition or removal of protection measures in the network (e.g., enabling/disabling 
a security function will be considered as an addition/removal security function). We do not support 
detailed settings of security devices such as filtering rules in a firewall. 

Future work will define the full service protection level as the overall protection of services for the 
entity due to several reasons: 1) to be aware of the general security level; 2) because actions to improve 
security for a service may have negative consequences to others (e.g., move of a security device), or may 
decrease the protection against other threats (e.g., replacement of a security device). 

Another task will consist in proposing guidelines for the protection effectiveness per type of security 
function. This parameter is very important in the proposed approach. Giving subjective value would ruin 
the effort to rationalize the RORI result. 
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